Analysis and Design of Echo State Networks for Function Approximation

The present design of echo state network (ESN) parameters relies on the selection of the maximum eigenvalue of the linearized system around zero. However, this has been found far from optimal for function approximation. This letter presents a function approximation perspective to better understand the operation of ESNs and proposes an information-theoretic measure, the average entropy of echo states, to assess the “richness” of the ESN dynamics. Furthermore, it provides a new interpretation of the ESN dynamics rooted in system theory as a combination of linearized systems where their poles move according to the input signal dynamics. With this interpretation, we will be able to a priori design ESNs with uniform pole distributions covering the frequency spectrum optimally. With adaptive read-outs, the designed ESN can be used as a general infrastructure to represent information in time. 1Correspondence address: Mustafa C. Ozturk, Computational NeuroEngineering Laboratory, Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, Tel: (352)392-2682, Fax: (352)392-0044